Papers by Lav R. Varshney
Transformer-based Causal Language Models Perform Clustering (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown great improvements in instruction-following capability through additional training for instruction- following tasks. |
| Approach: | They propose to use a Transformer-based causal language model to study instruction-following capabilities. |
| Outcome: | The proposed model learns task-specific information by clustering data within its hidden space, with this clustering process evolving dynamically during learning. |
DeepInsert: Early Layer Bypass for Efficient and Performant Multimodal Understanding (2026.eacl-long)
Copied to clipboard
Moulik Choraria, Xinbo Wu, Akhil Bhimaraju, Nitesh Sekhar, Yue Wu, Xu Zhang, Prateek Singhal, Lav R. Varshney
| Challenge: | Recent work shows that hyperscaling of data and parameter count in LLMs is yielding diminishing improvement when weighed against training costs. |
| Approach: | They propose to insert multimodal tokens directly into the middle of the model to bypass the early layers. |
| Outcome: | The proposed method reduces training and inference costs while preserving performance. |